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Executive Summary 



^ It IS almost impossible to overstate the magnitude of change taking place in the 
U.i. economy. The globalization of business, rapid changes in technology, and shifts in 
the demographics of the labor force have triggered a blizzard of books, reports and 
articles aU focused on a similar theme: America's woikforce must be retrained and 
upgraded to meet increasing international competition. The Commission on the Skills of 
the Amencan Workforce put it bluntiy in titling its report: America's Choice: high skills 
or low wages!. — 

America's current training effort seems enormous on the surface, but closer 
examination reveals a glaring gap in die system Analysis shows tiiat private employers 
mvest tiieir trammg resources in the most highly educated workers, while public efforts 
focus on die most disadvantaged Americans, many of whom are presendy out of d^e labor 
market Neglected in diis hodgepodge of public and private programs are established 
workers on whom die nation must rely to cope widi changing technology and international 
challenges. 

c V ^^"^^ Employment Training Panel (ETP) Program fills this unique niche 
bpecifically, ETP focuses on estabUshed workers who have recendy been displaced or 
who are m danger of being displaced. The question for policy mak<5rs is: Does diis 
pro^ work? Does it make a difference in die earnings and employment of die 
Calrformans it serves? What impact does die program have on die state's economy as 
a whole? This smdy's objective is to answer diese pressing questions by examining die 
performance of die ETP program. 

Does Training Work? 

The civil disturbances in Los Angeles, die worst in diis century, have focused the 
general public's attention on job training. A recent article in die Los Angeles Time^ 
noted diat "When asked die best way to prevent urban unrest, bodi whites and blacks 
Chose job-training programs over strengdiening die police by overwhelming margins." 

An assumption aboutjob training underlies diis renewed interest The assumption 
IS diat traimng works, and diat people who are trained find jobs related to dieir training 
earn more and are employed more dian if diey were not trained. Unfortunately twenty 
years of research on vocational training in a host of institutions does not uniformly 
support diis assumption. ^ 

In fact die most recent research on JTPA finds modest about $500, increases in 
income for adult female participants and no significant increases for male participants. 
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Two recent studies of displaced workers programs in Texas and New Jersey report that 
job search services alone or in combination with training yielded increased annual 
earnings of between $200 and $1^00. Its important to note that these programs did yield 
significant decreases in Unemployment Insurance payments to displaced workers. 

Recent research on GAIN participants in California revealed similarly modest 
impacts of between $200 and $400 in the first year after receiving services. 

Research on the impact of ETP training is limited to single study which examined 
the experiences of an eariy group ETP trainees who enrolled between January 1, 1983 and 
December 31, 1985. The study compared the before and after training earnings of 
participants, who were unemployed when they enrolled (called "new hires"), and trainees 
who were in danger of being displaced but were still employed (called "retrainees"). The 
study concluded that ETP training did have a significant impact on the eamings of 
trainees. The impact on new hires was estimated to be $5,976 after controlling for 
background and programmatic variables. The impact for retrainees was not statistically 
significant, after controlling for background variables, but estimated to be $2,003. 

While the study provided valuable insights into tlie impact of the program at that 
time, the ETP program has changed significantly since then. Most obviously, ETP has 
grown substantially in size. Rather than training three thousand people in several years, 
it now trains over 49,000 in a single year. During the previous period studied, 60% of 
ETP trainees were new hires, with only 40% being retrainees. Today less than 15% of 
ETP trainees arc new hires- 

Also the macro-economic context in which ETP trains has changed. During the 
previous study the California economy was moving out of a recession and into a period 
of rapid expansion. Today the opposite is true. The economy is in a period of declining 
growth after a long expansionary period. 

Overview of Approach 

Essentially this study used eamings and unemployment data from the California 
Employment Development Department and ETP program records to analyze the effects 
of ETP. The study measured eamings, weeks of unemployment and unemployment 
insurance payments in the year before and after training. The experience of both dropouts 
and completers were compared to assess the impact of ETP training and eamings while 
controlling for demographic and programmatic differences. Next it used the estimated 
effects of ETP on eamings to project the impact of the ETP program on the California 
Economy for one year and for longer periods. 
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Conclusions 



1. ETP training has a significant impact on the earnings of both new hire 
and retrainee completers. 

The results of this study clearly show that both new hires and rctrairiees who 
completed their program had significant increases in earnings that go beyond the rate of 
inflation. Looking at simple differences adjusted for inflation new hire completers earned 
over $10,900 more than dropouts, and retrainee completers earned over $3,500 more than 
dropouts. 

When a multiple regression analysis was used to measure the impact of completing 
training after controlling for dcnx)graphic and programmatic differences the impact of 
completing ETP training was still highly significant The estimated impact of completing 
training for new hires was over $7,700 and for retrainees over $4,000. 

We also calculated the simple average increase in adjusted earnings for people 
who completed ETP training. For new hires the increase was $7,400 and for retrainees 
it was $1,600. Anyway we cut it, completing ETP training had a significant and positive 
effect on real earnings. 

Economic theory leads us to conclude that these increased earning represent 
increased productivity for the individuals trained, and in the case of retrainees, increased 
productivity for the company that employs them. 

2. ETP training reduces the likelihood of unemployment for both new 
hires and retrainee completers. 

Trainees who completed the program experienced significantiy less unemployment 
after training than participants who dropped out of training. Retrainees who completed 
experienced about half the unemployment of dropouts, and new hires had only a third as 
much unemployment as dropouts. These dramatic differences may not be caused solely 
by ETP training; they may be due to other differences between dropouts and completers 
such as motivation and other characteristics. But the fact remains that the thousands of 
ETP trainees who completed training had significantiy less unemployment than those who 
dropped out. 



3. ETP appears to have a larger impact on earnings than traditional 
post*secondary vocational training offered in community colleges and 
proprietary institutions, GAIN services, JTPA Title 11- A or other state 
displaced workers programs. 
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As part of our analysis wc reviewed a wide range of studies that analyzed the 
impact of public training programs on participants earnings. These studies employed a 
range of research approaches in a variety of time periods, so their results are not exacdy 
comparable to our analysis on a dollar for dollar basis. But the comparison of the impact 
of ETP training, as we measured it, with other studies shows that ETP's impact is 
substantially larger. While the exact size of the difference is unclear, we believe the 
direction of the difference is significant and reliable. Studies of CETA and JTPA, and 
other state displaced worker programs show a consistent improvement in annual earning 
ot only several hundred dollars, while both this study and a similar previous study show 
the impact of ETP training to be several thousand dollars. 

Part of this gap is no doubt due to the fact that ETP trains established workers 
who have the background and skills to benefit from training, while many public programs 
serve more disadvantaged participants. But even when compared to other programs that 
train displaced workers, ETP showed a substantially larger impact However, we do not 
believe that this difference alone can account for the significant increases in earnings 
reported here. Our analysis convinces us that ETP's model of employer based training 
with stringent performance standards is powerful training model that produces consistent 
significant impacts on both earnings and the productivity of workers who complete the 
program. 

4. ETP has a substantial impact on California's economy, beyond the 
increased earnings of participants. 

Our analysis shows that the ETP training program included in this study had an 
impact on the state's economy that goes far beyond the increased earnings of participants. 
Depending on the assumptions used this impact ranged from $7 1 million to possibly over 
$1.1 billion in the first year after training. ETP has an impact on the economy through 
four mechanisms. First, it reduces UI payments by about $5 million dollars in the first 
year. Second, increased productivity, as measured by changes in earnings, accounts for 
an additional impact of over $66 million. Third, the increased productivity produces other 
economic activity within the state's economy which we estimate to be between $33 
million and $51 million. Fourth, assuming that the retrainees* jobs would have been 
temporarily lost, we estimate the value of saving the jobs form temporary loss is between 
$172 million and $257 million. If rctrainee jobs were to be permanentiy lost, the value 
of saving them is over $940 million a year. These results are particularly dramatic in 
light of the cost of these programs which was $64.8 million. 

From this analysis it appears the ETP training has a significant impact not only 
on the productivity of individuals but also on the productivity of companies. It seems 
logical to infer that if individual productivity increases so dramatically, it is likely that 
the overall productivity of the company increases, which in turn increases the 
competitiveness of the firm, and generates a multiplier effect on the economy. 
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This analysis points out that ETP is not just a training program but rather a 
powerful tool for economic development The issue for future ETP policy is can ETP 
training become a more effective tool by targeting its services on particular industries and 
companies. Tnis analysis provides a first step towards understanding how ETP training 
affects companies differently. 

5* Additional research is needed to understand the dynamics of ETP 
training's impact on companies and individuals. 

This research and the previous impact study provide a broad outline of the impact 
of ETP training. As the economy changes and EIP policy shifts, the impact of ETP 
training will also change. It is critical that ETP monitor its impacts empirically, so that 
the consequences of the new policies and the effects of shifting economic conditions can 
be accurately monitored 

In addition, this study suggests the impact of training is not uniform across 
industries or company type. It*s important that the panel understand how the impact of 
training varies so that it can eventually target training where it will have the biggest 
impact 

Recommendations 

1. ETP should maintain stringent performance standards for its contracts 
to insure that training is tied to employment 

The substantial impact of ETP on earnings and thus the overall economy are 
probably due to the powerful incentives created by the performance contracting standards. 
We believe maintaining these standards is a key to insuring the continued impact of the 
program. 



2. Develop policies to increase completion rates. 

While ETP has high completion rates (78% for retrainee and 69% for new hires) 
in comparison with similar public programs, ETP may be able to increase its completion 
rates. Since the bulk of benefits to both the individual and the economy as whole come 
from completing the program, increasing the proportion of completers will enhance the 
impact of the program. 

3. Develop targeting strategies that will lead to the greatest impact on 
California's economy. 

Previous research and reports have suggested methods for targeting ETP training 
that are ba^ed on the characteristics of firms and the need for training, or the likelihood 
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that jobs will be saved The results of this study suggest that the economic impact of 
ETP training will vary significantly depending on the company's industry and the ability 
of training to generate productivity increases. Our analysie suggests that ETP could 
develop targeting strategies that would maximize the economic impact of ETP training* 

4. Improve scope and quality of data collection of participant and 
program characteristics. 

A serious limit to this analysis was the fact that about 40% of the individual 
trainee records did not have the minimal demogr^hic data currcntiy collected by ETP, 
Policies that insure the collection of basic participant information arc necessary for 
effective research in the future, 

5. Track the impact of ETP training annually. 

ETP is under a mandate to evaluate die outcomes of ETP training. This makes 
sense because, as noted before, die impact of ETP training is likely to change as die 
economy changes and ETP policies are modified. This study shows that it is relatively 
easy to track the impact of ETP training by tracking the earnings and employment 
patterns of participants. We recommend that this study be replicated annually to provide 
policy makers and die public widi an objective assessment of the programs impact 

6. Study the impact of ETP training over time. 

Research suggests die impact of training lasts for 12 years, Whetiier diis holds 
true for ETP training is unknown. It is important for ETP to analyze its impact on 
participants and companies over time. It may be diat effects are stronger and last longer 
in certain industries for certain types of participants* This type of analysis would provide 
the panel with sound data for targeting training to create long term economic benefits. 

7. Conduct case studies of projects with exceptionally high and low 
changes in individual productivity. 

Case studies of projects where participants experienced large gains in earnings and 
thus productivity, and projects with minimal increases in earnings should be conducted 
to uncover die factors diat underlie die impact of training. Again, a detailed study of a 
few projects would identify the characteristics of projects tfiat produce the most powerful 
impact on earnings and productivity. It may also identify the characteristics of projects 
diat tend to diminish the impact of training. These cases would provide insights into how 
ETP funds could be targeted most effectively, to provide the maximum economic impact 
for funds spent 
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I Introduction 



We are living through a transformation that will rearrange the 
politics and economics of the coming century. There will be no 
national products or technologies, no national corporations, no 
national industries. There will no longer be national economies, at 
least as we have come to understand that concept All that will 
remain rooted within national borders are the people who comprise 
a nation. Each nation's primary assets will be its citizens' skills 
and insights. (Reich, 1991 pg.3) 

It is almost impossible to overstate the magnitude of change taking place in the U.S. 
economy. The globalization of business, rapid changes in technology, and shifts in the 
demographics of the labor maiket have triggered a bUzzard of books, reports and articles all 
focused on a similar theme: America's workforce must be retrained and upgraded to meet 
increasing international competition. The Commission on the Skills of the American Workforce 
put it bluntly in tiding its report: America's Choice: high skills or low wtiges!. The reports have 
much less agreement on how training should be done, and who should provide it Many of the 
audiors and commissions fall back on recommending expansion and reform proposals for 
traditional high school and post-secondary vocational programs. Others focus on reforming or 
expanding JTPA. A few note the lack of training for lower level workers in industry and propose 
various tax incentive schemes to encourage more employer based training. 

America's cuirent training effort seems enormous on the surface, but closer examination 
reveals a glaring gap in the system. Analysis shows that private employers invest then- training 
resources in the most highly educated workers, while public efforts focus on the most 
disadvantaged Americans, many of whom arc presently out of work. Neglected in this 
Does ETP Training Work? ^ 
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hodgepodge of public and private programs are established workers on whom the nation must 
rely to cope with changing technology and international challenges. The American Society for 
Training and Development estimates that private employers spend over $30 billion a year on 
training. But the training is limited in key ways* First, about $27 BiUion of the $30 billion is 
spent by large firms who make up only .5% of all employers (Camavale, 1986). Second^ other 
recent studies (National Center on Education and the Economy, 1992, Lillard and Tan, 1986) 
have found that the buUc of private corporation's training funds go to workers who already have 
high levels for education. For example, of the $30 billion spent on training each year, two-thirds 
goes to training workers with a college degree ( National Center for Education and the Economy, 
1992, pg.49). 

At the other end of the spectrum the federal Job Training Partnership Act (JTPA) focuses 
its efforts on the most disadvantaged workers. For example in California the 1991-92 JTPA 
budget for Htle 11, which serves disadvantaged adults and youth, is $264 million: the budget for 
Title m, which serves established workers displaced from jobs, is only $46 million. 

Thus, California's Employment Training Panel (ETP) Program fills a unique niche that 
is usually overlooked by both public programs and private training efforts. That is the training 
needs of established workers, the backbone of America's productive capacity. Specifically, ETP 
focuses on established workers who have recendy been displaced or who arc in danger of being 
displaced. The question for policy makers is: Does this program work? Does it make a 
difference in the earnings and employment of the Califomians it serves? What impact does the 
program have on the state's economy as a whole? The objective of this study is to answer these 
pressing questions by examining the performance of the ETP program. 
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Wc begin with a discussion of previous research on the impact of other public training 
programs. 

Does Training Work? 

The civil disturbances in Los Angeles, the worst in this century, have focused the general 
public's attention on job training* A recent article in the Los Angeles Times noted that "When 
asked the best way to prevent urban unrest, both whites and blacks chose job-training programs 
over strengthening the police by overwhelming margins." (Schneider, May , 1992, pg. ml) 

An assumption about job training underlies this renewed interest The assumption is that 
training, or vocational education more broadly, works, and that people who are trained find jobs 
related to their training, earn more and are employed more than if they were not trained. 
Unfortunately twenty years of research on vocational training in a host of institutions does not 
uniformly support this assumption. 

Starting m the 1970's, researchers began to examine the impact of publicly sponsored 
training on participants' earnings and employment Early studies, (for example Grasso and Shea, 
1979, Wilms, 1980), found tiiat publicly funded vocational programs had littie impact on 
participants* earnings or employment More recent studies in California of high school level 
vocational education found that, with few exceptions, students in vocational programs did no 
better in the labor market than simil'a students in the general education program (Stem, 
Hoachlander, Choy and Benson, 1986). 

In the most recent in-depth study of effects on earnings of attending public and private 
postsccondary vocational programs, Grubb (1991) found small effects even for individuals who 
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completed vocational Associates degrees, programs that often take two years. For example, his 



data show that after controlling for high school and demographic variables vocational Associate 

degree holders earned only $2,431 (in 1985 dollars) more than high school graduates. Reviewing 

his entire analysis, he concludes that: 

Above all, these results indicate that the returns within the sub* 
baccalaureate labor market are quite varied: certificates for men 
never have a return, certain fields of study have no effects on 
earnings, and those individuals with credentials from technical 
institutes and proprietary schools ma> earn no more than high 
school graduates. The simple faith diat more school automatically 
leads to higher earnings the lure of postsecondary education in 
general, and postsecondary vocational education in particular - is 
evidentiy not always justified. (Grubb, 1992, p, 18) 

Similarly, in 1973 the federal government began tracking the impact of CETA training 

on participants through the Continuous Longitudinal Manpower Survey. Samples of CETA 

participants were selected by the Bureau of the Census and interviewed periodically. These data 

were merged with Social Security records to form a longitudinal earnings record. A comparison 

group was formed with a sample from the current population survey. Bamow (1986) reviewed 

eleven major studies done with this data base. The studies examined the impact of CETA 

programs on various clients* earnings. Like the studies of post-secondary training, they found 

a wide variation in the impact, ranging from small negative effects to positive effects, ranging 

from $200 to $2,000 annually. Reviewing all tiie evidence, Bamow concluded that: 

".„a reasonable assessment of the impact of the CETA programs 
studied is that the programs probably had a modest impact of 
several hundred dollars on earnings for men, and a somewhat 
greater effect on women," (Bamow, 1987 pg,190) 
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More recent research on JTPA programs is liniited. A study measuring the impact of 
JTPA training on disadvantaged adults and youth, served by Tide II-A has just been published 
by the federal Department of Labor ( Bloom, etal. 1992), but significant research on the impact 
of JTPA displaced worker training remains to be done. However. Bloom, etal. (1986) using an 
experimental design again found only small impacts for adults assigned to JTPA training. The 
study found that women assigned to JTPA earned $539 more than women in the control group, 
during the first 18 months after training, a small but statistically significant margin. Men 
assigned to JTPA earned $550 more than men in the control group, during the first 18 months 
after training, however, this difference was not statistically significant. When the study looked 
only at those people who actually received services, as opposed to all assigned to JTPA, the gap 
between those served and die control group rose to $873 for women and $935 for men. 

Two recent studies have examined the impact of job service programs and training on 
displaced workers in Texas (Bloom, 1990) and New Jersey (Corson, etal., 1989 and (Anderson, 
Corson and Decker, 1990). These studies employed an experimental design found impacts of 
only several hundred dollars in the first year after receiving services. The New Jersey study 
(Corson, etal., 1989) examined three different types of services, job search, job search with a 
cash bonus for finding a job, and job search with training. Interestingly the treatment that 
involved training had the smallest impact on earnings, about $200, of all the services tested. In 
a longer temi foUowup of these workers (Anderson, Corson, and Decker, 1990) found that none 
of the three groups had any real increase earning. That is increases that exceeded inflation. The 
Texas study (Bloom, 1990), found substantiaUy larger impacts ranging from $673 for men to 
$1,148 for women, who received a mixture of job search and short term skill training services. 
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Tndning and employment programs focused on welfare participants had similar modest 
impacts. For example, a just released study of the California GAIN (Greater Avenues for 
Independence) welfare reform program (Riccio and Fricdlander, 1992) evaluated the impact of 
GAIN services on participants* earnings and welfare payments in the year after being served. 
The study used a careful experimental design with randomly selected participants and control 
groups. The study found only small impacts on earnings. The results found that GAIN single 
parent households earned an average of $271 more than a similar control groups. Participants 
from two parent households had larger gains of $375. In addition both groups experienced 
decreases in welfare paynnents, $281 dollars for single parent households and $420 for two parent 
households. 

Research on the impact of ETP training is limited to single study (Moore, Wilms, and 
Bolus, 1988). This study examined the experiences for an early group of 3,913 ETP trainees who 
enrolled in ETP programs between January 1, 1983 and December 31, 1985. The study 
compared the before and after training earnings of participants, who were unemployed when they 
enrolled (called "new hires"), and trainees who were in danger of being displaced but were still 
employed (called "retrainees"). The study concluded that ETP training did have a significant 
impact on the earnings of trainees. The impact on new hires was estimated to be $5,976 after 
controlling for background and programmatic variables. The rnipact for retrainees was not 
statistically significant, after controlling for background variables, but estimated to be $2,003. 

While die study provided valuable insights into the impact of die program at that time, 
the ETP program has changed significandy since then. Most obviously, ETP has grown 
substantially in size. Rather than training tiircc thousand people in several years, it now trains 
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over 49,000 in a single year. During the previous period studied, 60% of ETP trainees were new 
hires, with only 40% being retrainees. Today less than 15% of ETP trainees are new hires. 

Also the macro-economic context in which ETP trains has changed. During the previous 
study the California economy was moving out of a recession and into a period of rapid 
expansion^ Today the opposite is true. The economy is in a period of declining growth after a 
long expansionary period, 

ETP Today 

The Employment Training Panel was created by the state legislature in 1982 as part of 
several reforms within the Caiifomia unemployment insurance system. The ETP program was 
unique in that it did not set up anotiier set of training institutions, but instead relied on 
contracting with existing training institutions and most importantiy private companies to provide 
training. ETP training was also unique in that it was set up to be 100% performance based. This 
means that ETP only pays training providers for participants who complete training and are 
retained on a related job for 90 days. At the time this was a radical departure from most public 
training programs which paid training providers for the participants they enrolled, but seldom 
linked payment to outcomes. Since that time other training programs, notably JTPA, have moved 
to a more performance driven model. 

The primary original objective of ETP was to use its training fund to prevent or limit 
unemployment, by moving tiie unemployed quickly into new Jobs oi by saving the jobs of 
workers tiireatened with displacement. Since then the mission of ETP has been broadened. 
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In 1989^ Assembly Bill 28 expanded the focus of ETP. Besides reducing and preventing 
unemployment, ETP is also charged with improving the competitiveness of California by 
improving the productivity of its workers- The bill calls for increased focus on the needs of 
small businesses, tiiose witii less tiian 250 employees- The bill also requires tiiat ETP improve 
its coordination with other training agencies, 

ETP is supported by contributions to the Employment Training Fund, Each private, for- 
profit California Employer contributes .01% of wages subject to unemployment insurance tax to 
the fund This generated over $80 million for ETP in fiscal year 1991-92- In addition, funds and 
interest were carried over from previous years to create a total fund of over $167 million. In the 
last year ETP programs have been expanding- From July 1, 1991 to April 30, 1992, ETP 
approved 209 projects, budgeted at over $73 miUion, a 113% increase over the previous year 
(Employment Training Panel, 1992). 

Objectives of the Study 

ETP is an innovative training model that has attracted substantial attention from other 
states* Yet, empirical evidence about its performance is limited, A recent review of the 
Employment Training Panel noted tiiat the panel's experience is "replete with logical but untested 
theories and practices." It further observes that die panel is "mandated by statute to esublish a 
system to compare earnings of trainees before and after training". The report goes on to suggest 
tiiat the panel should get lists of projects witii substantial increases, decreases or no change in 
earnings, and should compare the characteristics of these projects. (National Center for Research 
on Vocational Education and The Center for Labor Research and Education, 1991) 
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The objective of this study is to go beyond this recommendation and attempt to measure 

the impact of ETP training on both the individual and the state's economy as a whole. More 

specifically this study attempts to answer the following research questions- 

1* To what degree did ETP participants^ earnings increase or decrease after 
training? 

2. To what degree did unemployment among ETP participants increase or 
decrease after training? 

3. How did earnings and employment outcomes vary by project? 

4. What demographic and project characteristics are systematically related 
to changes in earnings? 

5. What is the unique impact of completing ETP training on changes in 
earnings after controlling for Mfferences in demographic and project 
characteristics? 

6. What was the economic impact of ETP training on the states' economy, 
in terms of both increased productivity and unemployment insurance 
savings? 



n Methods 



Overview of Approach 

The methods used in the study build on an earlier study (Moore, Wilms, Bolus, 1988) that 
evaluated the employment and earning outcomes of an earlier gioup of ETP participants. 
Essentially this study used earnings and unemployment data from the California Employment 
Development Department and ETP program records to analyze the effects of ETP. This study 
measured earnings, weeks of unemployment and unemployment insurance payments in the year 
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before and after training. The experience of both dropouts and completers were compared to 
assess the impact of ETP training and earnings while controlling for demographic and 
programmatic differences. Next it used the estimated effects of ETP on earnings to project the 
impact of the ETP program pn the California Economy for one year and for longer periods. 

Study Population 

To avoid sampling error, the study attempted to include the universe of ETP participants 
who were enrolled in the 187 ETP programs that ended between July 1, 1989 and June 30, 1990. 
A review of ETP data bases found 46,946 cases that met this criteria, and for whom EDD could 
find records of earnings or unemployment insurance claims. We were able to include all these 
cases in some steps of the analysis. However a further search of the records found that a 
substantial number of these records were missing basic demographic data needed for more 
sophisticated analysis. 

As a result of the missing data problem, the final results of this analysis are based on a 
subgroup of 27,738 out of an initial population of 46,946. The reason the final analysis is 
restricted to this smaller group is that in order to conduct a complete analysis and create the 
multiple regression model described in the results section, we could only use cases which had 
complete data on variables included in the analysis. Since many cases were missing data on key 
demographic variables, this meant that a substantial number of cases had to be excluded from 
the final analysis. These data were unavailable because ETP contractors had not collected them 
on the required enrollment forms. Table 1 shows that 53.9% of the new hire cases and 59.3% 
of the retrainee cases were included in the final model. 
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Table 1 



Total Study Population and Final 
Population for Modeling 

Retrain New Hire Overall 

Total Population 45.023 1,923 46,946 

Final PopuIaUon 26,705 1,036 27,738 

Final Population 

as % of total 59.3% 53.9% 59.1% 
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Dropping so many cases raises the issue of whether or not the remaining cases are 
representative of the entire population. To assess the representativeness of the cases in the final 
model, we compared the included and excluded cases on type of industry and the employment 
and earnings outcomes on which we collected data. This analysis is presented in Appendix A. 
The results show that excluded cases were more likely to come from the durable manufacturing 
and finance sectors, and less likely to come from the retail and wholesale trade sector. In terms 
of outcomes excluded cases were similar to included cases. For example, average post training 
earnings for included cases was $27,413, for excluded cases $27,205 a difference of less than 1%. 
From this analysis we concluded that the remaining cases can reasonably be judged to represent 
the entire study population. 



Data Sources 

Four data sources were tapped to create a complete data set for each i jbject: 

ETP trainee files which contained demographic data, enrollment and 
termination dates for each participant; 

ETP project files, which contain additional data on each project; 

Unemployment Insurance (UI) records, from the California Employment 
Development Department (EDD), which contained the number of UI 
claims and the amount of UI payments made to each participant; and 

Quarterly earnings records, from EDD» which contain payroll tax repons 
with the quarterly earnings of each participant 
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In addition, wc used descriptions of each project included in the study and programmatic 
data to identify the Standard Industrial Classification (SIQ code of the employers served by the 
project, and to identify the training objective and type of skill of each project 



Analysis 

Data from these sources were merged to create a data base for analysis. Four measures 
of employment and earnings were calculated for each case: 

Annual earnings before training (unadjusted, adjusted and normalized): 
trainees* total earnings in dollars for the four complete quarters prior to enrolling 
in ET? training. These earning were "adjusted" for inflation using the Consumer 
Price Index for California, into 1990 dollars so that changes in real earnings could 
be measured The "adjusted" earnings were then normalized using the average 
California manufacturing wage to take into account changes in the demand for 
labor. All measures of earnings are reported separately. 

Annual earnings after training (unadjusted, adjusted, and normalized): 
trainees* total earnings in dollars for the four complete quarters after leaving ETP 
training. These earning were "adjusted" for inflation using the Consumer Price 
Index for California, into 1990 dollars so that changes in real earnings could be 
measured The "adjusted" earnings were then normalized using the average 
California manufacturing wage to take into account changes in the demand for 
labor. All measures of earnings are reported separately. 

Unemployment insurance claims and unemployment insurance payments 
before training: trainees* total number of weeks receiving unemployment 
insurance, and total dollar value of unemployment insurance payments in the four 
complete quarters before enrolling in ETP training. 

Unemp!dynient insurance claims and unemployment insurance payments after 
training: trainees* total number of weeks receiving unemployment insurance, and 
total dollar value of unemployment insurance payments in the four complete 
quarters after leaving ETP training. 
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The analysis presented here moves from the simple to the complex. We begin with a 
simple univariate comparison of new hires and retrainees to describe and compare the two 
populations- Next we analyze the employment and earnings records of dropouts and completers 
from both groups, by calculating the changes from the year before training to the year after 
training. 

Next we estimate the impact of training on changes in earnings by constructing multiple 
regression models for both new hires and retrainees. The objective of the models is to measure 
the unique impact of training on changes in earnings after controlling for demographic and 
programmatic variation among the trainees. A discussion of the technical aspects of the model 
is included with the model. 

Finally we estimate the economic impact of the entire ETP program on the California 
economy, under five different scenarios with differing assumptions. Again, the technical aspects 
of the economic impact analysis are included with the results. 

An analysis of the changes in income, UI claims and UI payments was also conducted 
for each project studied and delivered to ETP under a separate cover. 

Limits Of The Study 

While this study is rigorous and employs research methods commonly accepted in the 
social sciences, it has limits which the reader must consider while interpreting the results- 

First, like other quasi-experimental studies of this type, it lacks a randomly selected 
control group ~ that is a group of individuals who arc exactly like those trained by ETP 
programs, but who did not receive training. Instead this study compares participants who 
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completed training with dropouts. Beyond their nieasured background characteristics, which are 
controlled for statistically in the model, the degree to which dropouts and completers may differ 
is unknown. Completers may differ systematically from dropouts in their personal characteristics. 
Completers, for example, may be more motivated, or better adjusted to the training experience. 
Moreover, we do not know how much training dropouts may have received. They may have only 
attended a day of training, or they may have completed training but failed to complete the 90 day 
retention period on the job. The fact the dropouts may have received substantial benefit from 
training may minimize the measured impact of training. 

Second, as noted earlier a large proportion of ETP trainees could not be included in the 
regression models because ETP records lacked data on the participants backgrounds. While the 
available data indicates that these trainees were not radically different than those included, they 
may still differ in some systematic manner which may skew the results. 

A third limit may lie in the period in which earnings were measured. In his review of 
CETA studies, Bamow (1986) noted that earnings tended to drop before training, thus raising the 
question of whether taking income for the year before training accurately measures trainees' real 
labor market value. This issue is confounded in our study by the way we measured earnings and 
employment Earning and employment were measured for the four full quarters before enrolling, 
thus we did not capture some period of unemployment for each new hire, since we did not 
include partial quarters. For example, if a new hire enrolled in a program in February, we went 
back to December 31 and then took the four previous quarters, thus missing the period of 
unemployment in January and Febmary. 
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A final caveat must be applied to all multiple regression models like those used here. The 
models are capable of measuring unique relationships between variables as correlations, however 
these relationships arc not necessarily evidence of cause-and-effect relationships. Also these 
models arc subject to shrinkage, which simply means that when applied to other populations, 
such as trainees from 1992, they may have less predictive power. 

m Results 

This section begins by describing the characteristics of ETP trainees and comparing the 
characteristics of new hires and retrainees. It then compares the program outcomes for these two 
groups in terms of whether or not they completed their program. Next the economic outcomes 
of training are examined by comparing the before and after training earnings and unemployment 
insurance claims of dropouts and completers within each group. 

After this basic analysis we estimate the impact of training by developing regression 
models for retrainees and new hires that measure the impact of completing training on changes 
in earnings after controlling for demographic and programmatic differences. 

Finally, based on the earnings and employment outcomes for the individual trainees, we 
estimate the economic impact of HTTP training on the state's economy. 

Comparison of Retrainees and New Hires 

The typical ETP trainee was a rctrainee who completed his program. Demographicaily 
he was a 35 year old, married, white, male who had completed high school and about a year of 
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college. He was trained in the durable manufacturing or wholesale and retail trade industries. 

When we compared retrainees and new hires on the available demographic and 
programmatic data, we found many differences between the two groups. Retrainees, by far the 
larger group, tend to bti more advantaged than new hires. For example, jhey are more likely than 
new hires to be female, married, and white. Retrainees also tend to be slightly younger, have 
a bit more education, and are less likely to be veterans. Conversely, new hires are more likely 
to be male, single, and from a minority groups. They are also slightly older, have less education 
and are more likely to be veterans. 

In terms of program characteristics, retrainees arc more likely to be in the construction 
industry, non-durable manufacturing, wholesale and retail trades, and finance. New hires are 
more likely to be in durable manufacturing, transportation, service and the mixed industry group, 
indicating they were in a project that served a number of different industries. 

Projects were classified by the type of skills they trained in and by die overall objective 
of the project Retrainees were more likely to be in projects tiiat offered transferable skills 
(meaning that the skills can be used in other firms such as statistical process control), or in 
unique skills (skills tiiat can only be used at one firm, such as training on a customized inventory 
system). New hires were more likely to be in projects that offered training for entry into a 
particular occupation such as nursing or word processing. 

There were also differences in the objective of the projects. Retrainees were more likely 
to be in projects that focused on improving the productivity of an entire system, such as learning 
to master new inventory software, or projects that focused on improving the quality of a system, 
such as total quality management training. New hires tended to be in projects that focused on 
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raising personal productivity, such as projects that taught specific skills like word processing or 
numerical control machining. 

Retrainees had lower per participant costs than new hires. The average per participant 
costs for retrainees was $1,749 compared to $3,405 for new hires. 
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Table 2 



Profile of Retrainees and New Hires 
(Sample Trainees n=:27,741) 



New Hires Petrainees Chi Sq. 

N=1036 N=26,705 Sig. 



Gender qqq 

Male . 56.0% 50.2% 

Female 44.0 49.8 

Marital Status ,000 

Married 43.8 56.6 

Single 56.2 43.4 

Ethnicity .000 

Asian 14.7 11.8 

Black 17.6 5.8 

Hispanic 21.4 18.7 

Indian 1.5 0.06 

White 42.4 61.1 

Other 2.4 2.0 

Age: 

Average 36.8 35.3 .000 

Years 

of Educ 12.7 13.4 

Veteran 15.4% 10.8% .000 
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Table 2 (Continued) 



Profile of Retrainees and New Hires 
(Sample Trainees n=27,741) 



New Hires Retrainees Chi Sq. 
N=1036 N=26J05 Sig. 

Industry .000 



Agriculture 


3.1 


0 


Construction 


1.6 


2.5 


Durable 
Manufacturing 


37.2 


25.9 


Non-Durable 
Manufacturing 


1.5 


3.9 


Transportation 


1.3 


0.02 


Wholesale/ 
Retail 


0.06 


28.5 


Finance 


0.01 


16.6 


Services 


10.8 


9.8 


Unidentified 

or Mixed Industry 


43.9 


13.8 
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Table 2 (Continued) 



Profile of Retrainees and New Hires 
(Sample Trainees n=27,741) 



New Hires 
N=1036 



Retrainees 
N=26,705 



Chi Sq. 
Sig. 



Type of skill 

Basic Skills 

Transferable 
skiU 

Unique skill 

Occupation 

Mixed 

Project Objective 

Personal 
Productivity 

System 
Productivity 

Quality System 

Mix<xi 

Outcome 

Dropout 

Complete 

Cost per 
Participant 



0 

18.7 
1.0 
80.3 
0 



53.3 

45.0 

1.7 

0 

32.9 
67.1 

$3,405 



0.01 

27.1 
37.7 
34.8 
0.02 



18.9 

65.6 
15.3 
0.02 

20.3 
79.7 

$1,749 



.000 



.000 



.000 



.000 
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Program Outcomes 

The first step in assessing the outcomes of ETP training is simply to look at the 
proportion of trainees who completed their programs and were retained on a related job for 90 
days* Overall 78% of the 46,946 ETP participants in our study completed their training and were 
retained on the job 90 days* Table 3, which includes data on ail 46,946 participants, shows that 
retrainees were much more likely to complete their program (78*4%) than new hires (69.2%). 
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Table 3 

All Trainees by Eligibility 
Category and Outcome 



Retrain New Hire Total 

Complete 78.4% 69.2% 78.0 

(35^78) (1,330) (36,608) 

Dropout 21.6 30.8 22.0 

(9,745) (593) (10,338) 

Total 100.0 100.0 100.0 

(45,023) (1,923) (46,946) 
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Employment Outcomes 

The second outcome we examined was the number of unemployment insurance claims 
made by all 46,946 trainees before and after training, and how the pattern varied for trainees who 
completed and dropped out of training. Each claim represents one week for which trainees 
collected unemployment insurance. 

As Table 4 indicates, retrainees experienced very litde unemployment before and after 
training. Both retrainee completers and dropouts experienced, on the average, less than a week 
of unemployment before training. After training both completers and dropouts experienced a 
slight increase in unemployment after training, although, as expected, the increase was almost 
three times as large for dropouts. The increase in unemployment after training is probably due 
to the overall increase in unemployment throughout the state and nation during the period studied. 

The pattern for new hires was dramatically different Both dropouts and completers 
experienced nine to ten weeks of unemployment before training. After training, unemployment 
declined for both groups. But it fell only slighdy, 1.62 weeks, for dropouts and much more 
dramatically, 7.44 weeks, for completers. These data indicate that both dropouts and completers 
received some benefits from training. 

It should be noted that these figures probably underestimate the number of weeks 
unemployed prior to training for all groups. These data are based on the four full quarters prior 
to entering training. Thus if a trainee was eruroUed in the middle of three month quarter while 
unemployed, the six weeks he or she collected unemployment during that quarter would not be 
counted. 
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The amount Unemployment Insurance payment money actually received by trainees reflect 
the pattern described above. For rctrainccs payments increased slightly after training for both 
completers and dropouts. But, the average increase was far more dramatic for dropouts, over 
$180 compared to less than $80 for completers. 

Conversely, payments for new hires fell for both completers and dropouts. Completers 
had an average decline in Unemployment Insurance payments of over $900, compared to only 
$139 for dropouts. 
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Table 4 

Unemployment Payments and Claims 
by Dropouts and Completers 
(All Trainees N=46,946) 



Avg. UI claims 
bcfoirc training 



Avg.UI Claims 
after training 



Change 

in UI claims 



Retrainees 

Completers .32 

Dropouts c60 
New Hires 

Completers 9.79 

Dropouts 9.28 



.77 
1.73 

2.34 
7.65 



+.45 
+1.13 

-7.44 
-1.62 



Avg. UI Payments 
before training 



Avg. UI Payments 
after training 



Change 

in UI Payments 



Retrainees 

Completers $40.58 

Dropouts 77.66 
New Hires 

Completers 1,236.78 

Dropouts 1,121.71 



$117.86 
265.61 

310.65 
982.55 



+77.28 
+187.95 

-926.13 
-139.16 
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Earnings Outcomes 

A third key outcome we analyzed was the change in earnings from the year before 
training to the year after training. In this section we examine the earnings outcomes for two 
groups of participants. First we look at die earnings of all 46,946 trainees. Next we examine 
die earnings of the 27,741 trainees for whom complete data were available, called "sampled 
trainees". In each case we will break down the average earning of trainees before and after 
training for both retrainees and new hires, and within those groups, for diose who completed and 
who dropped out 

AH Trainees 

Among retrainees both completers and dropouts experienced an increase in earnings, but 
with a dramatic difference in the size of die average increase. Completers had an average 
increase ten times as great as die increase for dropouts. Completers experienced a gain of 
$3,897, a 15.8% increase over pre-training earnings. Dropouts had a gain of only $360, an 
increase of only 1.5% over pre-training earnings. 

The pattern for new hires was more dramatic. Completers experienced a gain of $8,263» 
an increase of 74.5% over pre-training earnings. Dropouts, on the other hand, had a loss of 
$1,265, a decrease of 11.0%. 
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Table 5 

Earnings Year Before and Year After Training 
for Dropouts and Completers 
(All Trainees N=46,946) 

Pre-Training Post-Training Change in % Change 
Earnings Earnings Earnings in Earnings 

Unadjusted 

Retrainees (N=45,023) 

Completers $24,641 $28,538 +$3,897 +15.8% 

Dropouts 24,263 24,623 +360 +1.5% 

New Hires (N=1^3) 

Completers 11,095 19,358 +8,263 +74.5% 

Dropouts 11,483 10,218 -1,265 -11.0% 
Adjusted 

Retrainees (N=45,023) 

Completers 26,706 28,321 1,615 +6.0% 

Dropouts 26,369 24,223 -2,146 -8.1% 

New Hires (N=1^23) 

Completers 12,057 19,456 +7,399 +61.4% 

Dropouts 12,442 10,153 -2,289 -18.4% 
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Part of the change in earnings reported above is due to inflation and thus does not 
represent a change in real earnings. To account for the impact of mflation we took the earnings 
for all participants and adjusted the earnings for inflation into 1990 dollars, using the Consumer 
Price Index for California (CPI), and then recalculated the average change in earnings. 

As Table 5 indicates inflation did account for part of the increase. For example, the 
increase for retrainees completers fell from 15.8% to 6.0%, and the change for retrainee dropouts 
went from a slight gain to a substantial loss -8.1%. New hires followed a similar pattern. 

It is important to note that the gains shown in adjusted earnings represent increases in 
earnings that go beyond inflation. This is particularly remarkable given that in California average 
wages adjusted for inflation were actually falling during this period. 

The key question is what do these substantial increases in earnings for completers 
represent Economists reason tiiat increases in earnings arc caused by two factors: either workers 
are working more hours or they are being rewarded for increased productivity. We know from 
Table 4 that retrainee completers actually experienced a slight average increase in unemployment 
so in this case it is logical to assume that the increase represents a substantial increase in the 
productivity of these workers'. In the case of new hires, who we know experience a significant 
decrease in unemployment, we calculate that 23.8 percent of the increase in earnings was due to 
working more hours and 76.2 percent was due to increased productivity. 
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Sampled Trainees 

The results for the sanq)led trainees, the group for whom we have complete data files, 
were very similar to the entire population, indicating only small differences between the groups. 
The only area in which this group diverges from the all trainees by more than a couple 
percentage points is in the case of new hires who dropped out These trainees experienced a 
substantially larger decline (19.0%) in unadjusted earnings than the larger group of all new 
trainee dropouts (1L0%X 

Table 6 presents the same information as Table 5 except that it is for the sampled 
trainees. Comparisons between the unadjusted and adjusted earnings in Tables 5 and 6 reveal 
the very minor differences between the population and the "sample" in earnings. In Table 6, 
though, we make one more calculation just to exclude the possibility that the increase in earnings 
observed for the trainees are part of a general trend in earnings. To test for this possibility we 
normalized the earnings, using the average manufacturing wage for Califomia witii 1985-87 as 
the index base. Table 6 shows how these adjustments affected the change in earnings 
experienced by the 27,738 sampled trainees. 

Normalizing the adjusted earnings using the average manufacturing wage for Califomia 
changed the earning figures siightiy but had no impact on the size of the increase or decrease 
experienced by trainees. 
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Table 6 



Unadjusted, Adjusted, and Normalized Before 
and After Training Annual Earnings 
(Sample Trainees N=:27,738) 



Pre-training Post-training Change In 

Annual Earnings Annual Earnings Annual Earnings Change 



UnAdjusted 

Retrauiees 
(N=26,702) 

. Completers $24,629 

Dropouts 23,600 

New Hires 
(N=1.036) 

Completers 11,733 

Dropouts 11,881 

Adjusted 

Retrainees 
(N=26,702) 

Completers 26,408 

Dropouts 25,371 

New Hires 

(N=l,036) 

Completers 12,606 

Dropouts 12,711 



$28,758 
24,138 

20,570 
9,628 



28,119 
23,576 



20,211 
9,492 



+$4,129 
+ 538 

+8,837 
-2,253 



+1,711 
-1,795 



+7,605 
-3.329 



+16.8% 



+23% 



+75.3% 
-19.0% 



+6.5% 
-7.1% 



+60.3% 
-26.2% 
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Table 6 (Continued) 



Unadjusted, Adjusted, and Normalized Before 
and After Training Annual Earnings 
(Sample Trainees N=27,738) 



Prc-training Post-training Change In % 

Annual Earnings Annual Earnings Annual Earnings Change 



Normalized 

Retrainees 
(N=26.702) 



Completers 27,307 
Dropouts 26,179 



New Hires 
(N=l,036) 



Completers 13,016 
Dropouts 13,158 



29,078 
24,323 



20,902 
9,823 



1,771 
-1,856 



7,886 
-3,335 



+6.5% 
-7.1% 



+60.6%. 
-25.3% 
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Another way to portray the different experiences of dropouts and completers is to examine 
the gap between their earnings. Table 7 shows this gap for unadjusted, adjusted and normalized 
wages. The data show a consistent gap of about $3,500 between retrainee dropouts and 
completers. A much larger gap exists between new hire dropouts and completers, of between 
$10,000 and $11,000. 
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Table 7 

Difference Between Dropouts and Completers 
In Change In Annual Earnings 
(Sampled Trainees N=27,738) 



Difference In Change 
In Earnings 



Unadjusted 

Retrainees $3,591 

New Hires 11,090 
Adjusted 

Retrainees $3,506 

New Hires 10,934 
Normalized 

Retrainees $3,627 

New Hires 11,221 



Does ETP Training Work? 



47 



34 



Again, as noted earlier, changes in earnings represented a substantial increase in the 
productivity of these workers, particularly among the retrainces. To a lesser degree it represents 
an increase in the hours worked for new hire trainees. 

Multiple Regression Analysis 
Overview 

The earlier analysis showed that completers of ETP training earned significantiy more 
than dropouts. A question remains, however was this difference in earnings due to training, or 
is it due to differences in the characteristics of completers and dropouts and the programs that 
served them? Perhaps completers were more highly educated than dropouts, or maybe completers 
worked in industries with higher wages. 

Multiple-regression analysis provides tool for measuring the unique relationship between 
changes in earnings and completing ETP training while controlling for differences in 
demographic and project characteristics. The multiple-regression analysis presented here has two 
objectives: 

to identify the unique impact of completing Gaining on changes in 
earnings; 

to identify the unique relationship of demographic and program variables 
on changes in earnings. 
The variables examined in this analysis are listed in Figure L 
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Figure 1 

Variables Included In Multiple-Regression Analysis 



Coiiq)leting Training 

Demographic Variables: 
age 

gender 
marital status 
education level 
military service (veteran) 
ethnic group 



Program Variables: 

cost of the training (per completing trainee) 

number of trainees in the program 

industry for which they arc trained 

type of training 

basic skills (language, math, writing, etc) 
transferable skills (skills useful in various jobs) 
unique skills (useful mainly in a particular job) 

training objective 

personal productivity enhancement 
system productivity enhancement 
product quality enhancement 
qualifying trainee for a new occupation 



Method 

The specific dependent variable which the regression model attempts to predict is the 
adjusted change in earnings. Recall that this variable is adjusted to eliminate any effects of 
inflauon on earnings. This adjusted earnings variable is obtained by dividing the reponed 
earnings by the Consumer Price Index for California wage earners; it is sometimes called ''real 
earnings" but here it is called adjusted earnings. 



Does ETP Training Work? 

36 



erIc ^'^ 



The change in adjusted earnings was regressed on the variables listed before to determine 
the direction and magnitude of each variable's influence on adjusted earnings. 
Two separate regression analyses were run, one for the "new hires" and one for the "retrainees". 
This allowed for the possibility that some of the variables above would affect the group of new 
hire trainees differendy than they affected the retrainees, which was the case in several instances. 

The analysis produced three key statistics which arc reported here. First, the statistic 
indicates the percent of the total variation in adjusted change of earnings which is explained by 
the variables in the model. This statistic gives an overall indication of the power of the analysis. 

Next is a regression coefficient, expressed in dollars. This statistic indicates the unique 
contribution of each unit of the variable to adjusted changes in earnings. This measure may be 
positive or negative. Thus, a coefficient of -$149 for age, in the new hire analysis, indicates that 
for every year older a person is, their change in earnings would decline $149, Similarly, the 
$7,487 coefficient for being a high school graduate indicates that the unique effect for completing 
high school, as the highest level of education attained, was an increase in earnings of $7,847. 

Finally, in parentheses under the regression coefficient is the confidence interval. This 
figure, also in dollars, indicates that two-thirds of cases fell within this range of the coefficient. 
For example, in the case of age for new hires, the confidence interval indicates that for two-thirds 
of the cases the actual coefficient was between -110 and -188. 

For each coefficient we indicated the statistical significance of the relationship with a 
series of asterisks. Statistical significance indicates the probability that the estimated regression 
coefficient is not equal to zero. Specifically, the regression analysis uses the "confidence 
intervals", described before, and the estimated coefficient to calculate levels of probability. The 

Does ETP Training Work? 37 

50 



probabiUty indicates the likelihood that the "true coefficient" Ues within a specific confidence 
interval around the estimated coefficient If that confidence interval does not contain zero, then 
the coefficient is said to be significantly different fix)m zero at the level of probability indicated. 
Most studies use a five and/or a one percent confidence interval, indicating there is a oiie or five 
in a hundred chance that zero is contained in the confidence interval. To minimize the possibility 
of identifying a coefficient as not equal to zero when it really is, this regression analysis uses a 
confidence interval which corresponds to one percent or less; this means that the coefficient is 
considered to be significant only if the probabUity is 99 percent or greater that the coefficient is 
not equal to zero. 

Table 8 contains the results of the regression analysis. The coefficients marked with an 
asterisk arc highly significant, with the probabiUty of 99 percent or greater that the coefficient 
is not equal to zero. The coefficients without asterisks can be considered essentially zero -- 
indicating that the identified variable has no statistical relationship with the adjusted change in 
earnings. 

Regression Results: Power of the Analysis 

Table 8 reports the estimated coefficients for the regression analysis (with the standard 
errors of the estimates in parentheses below). The number of observations (N) and the (the 
observed variation explained by the analysis) are shown for the new hires and the retrainees. 

As the table 8 indicates, the new hire model explained about 29% of the variance in the 
adjusted change in earnings, while the retrainee analysis explained 5% of the variance. Both 
these amounts are highly statistically significant and it is unlikely they are due to chance. What 
it docs indicate is that a large proportion of the variance may be related to factors not measured 
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by this study, such as the personality characteristics of participants or the type of trainer, or may 
be simply random variation. Additional data would have probably added to the explanatory 
power of this analysis. For example, a more refined classification of the industry in which 
training occurred or the occupational classification for which the people were trained might have 
increased the explanatory power. Training results may vary systematically by the type of trainer 
or by other parameters that were not measured. The amount of variation explained also would 
have been higher if complete data were available for all trainees. 

The percent of variation explained for the rctrainee equation is well within the range of 
similar studies, (see for example Grubb (1991)), and the new hire regression has a much higher 
explained variation (R^) than most similar studies. This is probably do to the more homogeneous 
experience of the new hires relative to the rctrainces. The new hires were all unemployed at the 
initiation of training, and self selected to participate, whereas the retrainees were all currently 
employed but in a variety of industries and occupations. Bear in mind, though, that the lower 
explained variation of the retrainee equation in no way detracts from the significant influence that 
the examined variables had on the change in adjusted earnings. 

For now, the highly significant influence of some variables on change in adjusted earnings 
is reported and account is taken of the low explained variation of the regression in the application 
of these results to determine the impact of ETP on the California economy. 
Impact of Completing 

The key question this analysis addresses is, what was the impact of completing ETP 
training on changes in adjusted earnings? 
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Our analysis indicates that the value of completing ETP training for new hires, 
independent of all other variables, was $7 JOS of increased earnings adjusted for inflation. For 
retrainees the increase was $4,037. In both cases the coefficient is highly significant, with 
narrow confidence intervals, indicating that these differences are not due to chance. 

In interpreting these coefficients we also need to consider the value of the intercept The 
intercept value at the top of table 8 can be carefully interpreted as revealing the influence of 
enrolling in training (without completing) when evaluated at the zero level of the other 
independent variables. The intercept for new hire regression is not significandy different from 
zero, but the coefficient for completing training is large and very significant. This means that 
enrolling in training probably had no significant influence on the earnings of new hires but that 
completing training was to be expected to boost a new hire's eamings by $7,700 in the first 
complete year after training. 

In contrast, the retrainee regression shows that both enrolling in training (as indicated by 
the significant intercept) and completing training had significant impacts on eamings. 
Completing training added $4,000 to retrainee eamings. Enrolling in training (the intercept) 
contributed about $2,100 to first year eamings for the retrainees (after adjustment for the average 
age and program size). These effects may be viewed as additive, generating an expected $6,100 
for the retrainee who completes training. 

We hypotiiesize that new hire trainees and retrainees had different outcomes because they 
had different post-training experiences on the job. A retrainee who enrolls and completes tiainmg 
becomes more productive by virtue of the training iti^elf and the upgraded job. A retrainee who 
docs not complete training may still end up working with die other retrained people and be more 
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productive (and higher paid) as a result The new hire who does not complete training is unlikely 
to end up working with others who are newly trained or retrained. Therefore the entire effect 
of tzaining for the new hire depends upon the successful completion of the training. 
Regression Results: Demographic Variables 

Several demographic variables proved to have a significant relationship with changes in 
earnings* Table 8 provides data for each variable in the new hire and retrainee analysis. 

The age of the trainee had a significant influence on the change in earnings for both new 
hires and retrainecs- For both new hires and retrainees, the change in adjusted earnings fell about 
$150 for every year older (over 18) the trainee was. This indicates that training is slightly more 
productive for younger workers. 

The gender of the trainee was a significant influence only for retrainees, where being 
male was associated with about $675 more in the change in earnings. Gender had no influence 
on the change in earnings for new hires. 

The maiital status of new hires likewise had no effect on their change in earnings. 
However, for retrainees, being married reduced the expected change in earnings by about $500. 

The education level of the trainees had similar directional effects for new hires and 
retrainees but the effects were very different in magnitude. In both cases, a high school degree 
or less had no influence on the change in earnings, but having some college or a college degree 
had a significant and positive impact. Either some college or a college degree added around 
$7,500 to the change in first year earnings for new hires. Retrainees experienced a $1400 to 
$1,900 increase in earnings for some college or a college degree. 
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Ethnicity had no significant impact on the change in earnings for either new hires or 
retrainees. This result held for all ethnic groups* It is interesting to note that none of these 
coefficients met even a 5 percent level test for significance. Under a 10 percent level test, which 
is virtually never accepted in the research literature, being Hispanic would have added to the 
change in earnings for new hires, and being Asian would have contributed positively to the 
change in earnings for retrainees. 

Military service status - being a veteran - had no significant effect on the change in 
earnings for new hires, but was associated widi about $900 less in the change in earnings for 
retrainees. 

Regression Results: Program Variables 

The cost of the program per participant had no significant influence on the change in 
adjusted earnings for eitiier group. This mean that more cost v/as not a significant factor in 
detcmiining die change in earnings for eitiier group. This does not mean tiiat in any individual 
case, a lower cost program would have produced die same adjusted earnings change as the 
program tiiat was run. It simply means that die variations in costs between die programs that 
were run produced no systematic differences in the change in earnings. 

The size of the program did produce a significant difference in earnings for die retrainee 
group. The negative and significant coefficient for diis group means diat die retrainees 
experienced $0.44 less in the change in adjusted eammg for every additional person in the 
program. For retrainees, smaller programs produced a larger increase in, adjusted earnings. At 
diis point, witii out more specific project data, it is difficult to explain why diis relationship 
would exist 
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The industry in which the people were trained had a large and positive impact on new 
hires if they were in the transportation and utilities industry group. While die coefficient for this 
industry at $17,500 is very large, it is interesting to note that die size of its standard deviation 
indicates that this coefficient is significantiy different from zero, but is not signiScantiy different 
from $1500. Retrainecs experienced significantiy lower changes in earnings if they were in 
durable manufacturing ($-13CK)), wholesale and retail trade ($-2100), or services ($-1400). These 
are probably not unexpected results given die declining demand for workers in durable 
manufacturing, and the relatively low pay levels in trade and services. 

The type of training provided had a significant influence on the change in earnings in only 

one case. Among the new hires, training people witii transferable skills resulted in a significantly 

lower change in earnings. This is a somewhat puzzling result Transferable skills are supposed 

to be skills, such as numerical machining, diat would be valuable to other potential employers, 

so one would expect die change in earnings to be greater (if anydiing) in diis case. Preliminary 

analysis shows, however, that there were only 194 new hires enrolled in transferable skill 

programs, and that about one half of them dropped out before completing training. It is possible 

that this variable picked up die influence on earnings of some particular projects, rather than the 

influence of transferable skills training as such. This suggestion is reinforced by the absence of 

a significant effect of transferable skills training for retrainees. 

« 

Finally, die training objective had a slightiy negative effect on the change in earnings of 
retrainees in the cases of personal productivity or system productivity objectives relative to the 
objective of product quality. This simply means that retrainees in programs designed to enhance 
product quality experience about $800 more change in earnings relative to other retrainees. 
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Table 8 



Regression Models For Predicting 
Change In Adjusted Annual Earnings 



New Hire Retrain 
(N=1035) (N=26,701) 
Adjusted R2=.289 Adjusted R2=.048 



Intercept -$14397 $5,130* 

(6,959) (1,735) 

Complete Training $7,705*** $4,037*** 

(829) (179) 

Age -149*** -167*** 

(39) (7.6) 

Gender 356 657*** 

(Male) (957) (162) 

Marital Status -318 -495** 

(Married) (726) (149) 

Education 

< H.S. 4,232 578 

(2,839) (588) 

H.S. Grad 7,847* 868 

(2,648) (508) 

Some Coll. 7,419* 1,400* 

(2,694) (514) 

CoU. Grad. 7,860* 1,920** 

(2,825) (522) 
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Table 8 (Continued) 

Regression Models For Predicting 
Change In Adjusted Annual Earnings 



New Hire Retrain 
(N=1035) (N--^26,701) 
Adjusted R2=.289 Adjusted R2=.048 



Ethnicity 

Black 1,358 -68 

(1,037) (309) 



Hispanic 1,757 68 

(974) (197) 



Indian 3,234 -360 

(2,939) (918) 

Asian 1,652 447 

(1,135) (233) 

Other 2,028 514 

(2,331) (516) 



Veteran -461 -929** 

(1,065) (256) 

Program Characteristics 

Cost Per Trainee 1.43 -.21 

(.69) (.16) 

Number in Program .32 . 44*** 

(.39) (.07) 
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Table 8 (Continued) 



Regression Models For Predicting 
Change In Adjusted Annual Earnings 



New Hire Retrain 
(N=1035) (N=26,701) 
Adjusted R2=.289 Adjusted R2=.048 



Industry 

Agriculture 6,372 NA 
(6,051) 

Construction ' 12,418 -721 

(6,714) (555) 

Durable Manuf. 13,880 -1,344*** 

(5,751) (343) 

Non-Durable Manuf. 13,612 -509 

(6,511) (483) 

Transportation, 

UtiUties 17,507* -72 

(6,612) (1,580) 

Wholesale/Retail 7,692 -2,052 

(5,396) (293)*** 

Finance,Insurance 

Real Estate 8,856 -19 

(15,711) (316) 

Service 7,032 -1,358*** 

(5,871) (342) 
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Table 8 (Continued) 

Regression Models For Predicting 
Change In Adjusted Annual Earnings 



New Hire Retrain 
(N=1035) (N=526,70l) 
Adjusted R2=.289 Adjusted R2=.048 

Type of Skill 

Basic NA -2,946 

(2.509) 

Transferable -8,348*** -249 

(1,153) (1,557) 

Unique -8,100 -341 

(5,928) (1,568) 

Occupational NA 334 

(1571) 

Training Objective 

Personal Prod. .-1,754 -888* 
(3,028) (308) 

System Prod. 2,739 -804* 

(5,874) . (269) 



* = Significant at .01 
** = Significant at .001 
*** =Significant at .0001 
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Estimating the Impact of Training on the California Economy 

The impact of ETP training on the California economy is the difference between what did 
occur in the California economy with ETP training programs and what would have occurred in 
their absence. Depending on the assumptions used, the estimates of ETP*s impact range from 
about $70 million to over $ 1.1 billion in the first year after training. 

Our analysis of ETP's economic intact has two steps. First, we analyze the economic 
dynamics of ETP training to identify the components through which ETP training programs can 
affect economic activity. Next, we estimate the impact of ETP training on each of the 
components and aggregate the effects, to get the overall impact of ETP on the state's economy. 



The Economic Components 

The economic impact of ETP training has four components. 



Component 1: Unemployment Insurance Savings 

Component 2: Increased productivity of trainees (measured in earnings) 

Component 3: Multiplier effects of increased productivity 

Component 4: Value of jobs saved. 



Using these four components we estimate the economic impact of ETP training on the 
California economy for the first year after training. In the following section we explain how 
ETP training affects each component and how we arrived at our estimate of the impact. 
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Component 1: Unemployment Insurance Savings 

One goal of ETP training is to reduce unemployment among participants* To the degree 
that ETP succeeds in reducing unemployment, it saves the state money by reducing 
unemployment insurance claims* The new hires are unemployed at the time of training and 
presumably their unemployment would continue for some period of time if they were not trained 
and placed by an ETP program. The retrainecs may also be saved from suffering substantial 
unemployment by ETP retraining, since ETP requires that retrainecs be in jeopardy of losing their 
jobs* Presumably, ETP training saves the retrainecs from this near certain unemployment and 
therefore lowers unemployment insurance outlays* 

We have developed two estimates of the UI savings attributable to training. Our first 
measure of the impact of ETP training on Unemployment Insurance (UI) claims is the difference 
between the post-program UI claims of completers and the post-program claims of dropouts. The 
assumption here is that without ETP training (including placement), completers would experience 
the same level of UI claims as dropouts* Thus the difference between the UI claims of 
completers and the dropouts times the number of completers is our estimate of the UI savings 
attributable to ETP training* This calculation is made for both retrainecs and new hires to obtain 
the estimate of UI savings, shown in Table 9. 

Our second estimate for UI savings makes the reasonable assumption that the retrainecs 
would experience some average period of unemployment if they had not received training. This 
alternative assumption derives from the ETP program requirement that the retrainecs be in 
jeopardy of losing their jobs if training is not provided. If this criterion is accurately applied, one 
would expect tile retrainecs to have become unemployed if ETP training had not been provided. 
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An estimate for the doUar value of UI claims in this case can be calculated by multiplying the 
average weekly claims times the average period of unemployment 

We have three slightiy different estimates of the average period of unemployment (as 
shown in Scenarios 4A, B and C in Table 9). Estimate A is based on our study population and 
assumes tiiat the retrainees would have had the average number of weeks of UI claims 
experienced by tiie new hires eitiier for tiie year before training, or by new hire dropouts in the 
year after training: tiiese two estimates average 9.5 weeks. This average is likely to 
underestimate tiie true period of unemployment for tiiese workers since our data only include the 
last complete quarter before entry into a training program, and also only include the tlrst 
complete quarter after participation. Thus tiiis estimate omits tiie UI claims in die quarter in 
which tiie unemployed person enroUed in training and tiie UI claims in tiie quarter die person 
dropped out of training. The 9.5 week estimate is used only because it represents the lowest 
available estimate of an average unemployment period. 

Two otiier reasonable estimates of average unemployment periods arc 10.6 weeks or 14.2 
weeks. The 10.6 week average is tiie 1990 Bureau of Ubor Statistics estimate, as determined 
in tiieir montiily labor market survey for California and is shown in Scenario 4B. The 14.2 
weeks is tiie acmal average period for UI claims for 1990 in CaUfomia and is shown in Scenario 
4C.' 



The average lengtii of unemployment from tiie household survey which covers all workers 
(scenano 4B) is shorter tiian tiie state-wide average number of weeks UI collected because 
workers covered by unemployment insurance tend to be out of work for longer periods than the 
average for all workers. These 1990 averages were obtained from die California Employment 
Development Department 
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Component 2: Productivity Increase 

The productivity of ETP trainees may increase because they either produce more per hour 
worked, or because they work more hours in a year. Any training program that increases the 
marketable skills of the participants creates additional economic capacity to produce. ETP 
programs may have an advantage over other training programs, though, because of the placement 
requirement in ETP contracts. Other training programs create the potential to produce more; that 
potential is realized when the newly trained workers are placed. ETP contracts require placement 
as a condition of training, so when ETP training is completed the economic potential of the 
enhanced productivity is realized. Thus, die impact of ETP training is realized immediately in 
increased output for die California economy due to the enhanced productivity of the trainees who 
complete the program. In addition, because trainees are now more skilled, they may have the 
opportunity to work more hours by rvoiding unemployment 

Unfortunately, directly measuring an individual's change in productivity is difficult and 
expensive. Direct measurement of productivity requires accurate physical output and input data 
for the periods before and after training. Such data are generally not available, and ETP projects 
are no exception. Even if that data were available, direct measurement of individual productivity 
is complicated because newly trained workers typically work with different equipment, different 
materials, and differentiy skilled people tiian they did before training. It is difficult to separate 
a single worker's change in productivity from the contribution made by the new equipment, new 
materials, and differentiy skilled workers. 

In this project, as in most studies, the change in earnings is usedTas an indicator of the 
change in worker productivity. Standard economic theory implies that workers are paid the value 
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of what they contribute to production (their marginal product). Accordingly, an increase in 
productivity should show up in an increase in workers' earnings. The advantage of using 
earnings as an indicator of productivity is that earnings data is much more available than physical 
output data and earnings data is reported on an individual basis. Because records of the physical 
change in production levels do not exist for ETP projects, we used the changes in earnings data 
as an indicator of changes in productivity. 

The regression analysis produced estimates of the impact of completing ETP training for 
both new hires and retrainees. Those estimates are $7,700 change in adjusted earnings for new 
hires and $6,100 change for retrainees. While these estimates might be adjusted a little more to 
obtain the statistically estimated average for the population (by multiplying the er,timated 
coefficients times the mean values of the variables), it seems unlikely that the regression analysis 
is going to produce an estimate of less than a $5,0(X) increase in adjusted earnings for those who 
complete training in either group. A minimum of around $5000 increase in adjusted earnings 
for completion of the training program is a large estimated impact, compared with other public 
training program and speaks well for the ETP program. However, the projection of overall 
program impact on the basis of a regression analysis that explains only 5 percent of the variation 
in the change in adjusted earnings (in the equation for of retrainees) would cause concern for 
some. 

Consequentiy, the estimate for the overall impact of ETP training on earnings is taken 
from the actually observed average change in adjusted earnings for program completers. This 
change in earnings number was available for all of the 36,608 people who completed training in 
the sample period. The overall total was comprised of 35,278 retrainees and 1 330 new hires who 
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completed training. The ubulated average increase in adjusted change in earnings for retrainees 
who completed was $1,615 while that of new hire completers was $7,399. For estimation 
purposes the $1,615 change for retrainees was rounded lo $1600, while the change for new hires 
was rounded to $7,400. 

The use of these numbers as a basis for estimating the overall impact of ETP on the 
California economy satisfies several objectives. First, since these averages are based on the full 
number of training completers in the sample period, there is no question of the representativeness 
of the smaller group (for whom there were complete demographic and program records) used in 
the regression analysis. Secondly, the use of the actual reported change in earnings for 
completers rather than a projected changed based on regression equations eliminates any question 
pertaining to the influence of the smaller explained variation on the estimated impact of ETP. 
Finally, since the actual reported average change in earnings is the smallest of die three available 
estimates^ of the training on earnings, and using the smaller estimate is die more prudent 
approach. In Table 9 we simply multipUed the average adjusted change in earnings for 
completers times the number of completers, for new hires and retrainees, to get the total change 
in productivity for the trainees.' 



^The third estimate is the difference in the change in earnings between the completers and 
die dropouts. The differences in adjusted earnings is $3,506 for retrainees and $10,934 for new 
hires as reported in Table 7. This gap between completers and dropouts is larger than the 
estimates used in our projections because die dropouts actually experience some decrease in 
adjusted earnings. Some researchers advocate tiiat diis gap is die appropriate measure of the 
effects of training. 

' Some researchers argue diat changes in wages underestimates die increase in productivity 
because some common pay practices (see for example Bishop, 1985). If diis is correct then these 
estimates understate die real impact of ETP training. 
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Component 3: Multiplier Effect On Increases Productivity 

ETP also contributes to the California economy by enhancing the competitiveness of the 
businesses through upgraded jobs and worker skills in the rctrainec programs. California has the 
largest economy in the nation and is a major trade center for the Pacific Rim. Consequently, 
businesses that produce, warehouse, transport, or sell nationally and internationally traded goods 
and services face rigorous domestic and international competition. ETP retraining programs 
upgrade both jobs and workers and this strengthens the competitive advantage of California 
businesses relative to tiieir out-of-state or international competitors. An improved competitive 
advantage allows these California businesses to maintain or improve their market shares, this 
means increased sales for these businesses and increased activity for die California economy. 

Standard regional economic analysis delineates economic "basic activities" from "service 
sector" activities. The businesses that operate in these competitive national and international 
markets arc the "economic base" of the rcgion*s economy. 

According to the regional theory, these "basic" activities generate the jobs in the "service 
sector" of the local economy. The "services sector" includes all the businesses that are suppliers 
to locally-situated industry and households, or put another way, businesses which respond to 
locally-generated demand for goods and services. The service sector is typically identified with 
the local retail trade, local business services, most personal services, and local government 
activities. In contrast, businesses that supply demands generated outside the local economy are 
part of the economic base. These include: the portion of local hotel, restaurant, entertainment 
and retail trade that services visitors; government activities funded by non-local sources; 
interregional financial, insurance, transportation, and utility networks; and along with all export- 
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oriented activity.* The multiplier effect accounts for the new economic activity created in the 
service sector as local suppUers respond to the higher demands from the basic sector. This 
suggests that ETP could be most effective in fostering growth in California by concentrating its 
training programs in the basic sector industries. Training in these industries would increase 
output both direcdy (the productivity effect) and indirectiy (the multipUer effect). 

Note that training among the "retrainees" is likely to generate these multiplier effects but 
training among the "new hires" is not Retrainee programs involve an upgrading of the job as 
well as the workers' skiUs. By upgrading jobs, businesses gain the capacity to produce more and 
therefore will order more goods and services from its suppliers. In addition since retrainees 
become more productive they earn more and thus spend more in the local economy generating 
additional economic activity. 

In contrast, ETP trains "new hires" because these people arc currcndy unemployed and 
require training to qualify for a job that is already available. An important feature of new hire 
training is die presumption that die job is available, and tiiat if die new hire trainee did not fill 
tiie job, someone else would. Thus, die consistent assumption for die new hire trainee is diat die 
training increases die productivity of die newly-trained worker, but does not increase the level 
of production in die host firm. This assumption -- diat no new or more productive jobs are 
created in training new hires - limits die benefits of new hire training to the increase in 
productivity experienced by die worker. That is, die benefit of new hire training is only the 
increased productivity of our work force and the productivity of California businesses are not 



Basic industries in die California economy have been researched and identified by the 
Center for die Continuing Study of die California Economy (CCSCE) in Palo Alto. 
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enhanced through new hire training. This assumption is conservative but it is justified by die 
nature of nwst of die new hire training projects. This conservative assumption is maintained 
diroughout die series of estimates. Multiplier effects are assumed only for die retraining 
programs where job upgrades are present 

The multipliers used in diis study were developed from die IMPLAN system specifically 
for diis impact analysis. The IMPLAN system can- generate output, income, and employment 
multipliers, among odicrs. A more complete description of die IMPLAN multipliers appears in 
Appendix C along widi die multiplier values for die various industries. We used die set of output 
multipliers for California's basic industries which averaged about 1.9. This multiplier value 
indicates diat for each dollar of new activity in die basic sector diat 9/10 of an additional dollar 
of activity is created in the local economy. 

It is common to apply die multipliers to changes in basic activity to determine die effect 
of diat change on total activity. We used a list of industries diat have been determined to be 
basic industries in California to determine die hkely extent of retraining programs that are in 
basic industries. Our preliminary analysis of die ETP projects in terms of correspondence to the 
basic industry list suggests diat about 65 percent of die retraining occurred in the basic 
indusaies'. We dierefore applied die multiplier of 1.9 to 65 percent of die increase in eamings 
generated by retraining to obtain diis estimated multiplier effect on die economy. 

Job upgrading in die service sector retraining programs could also have multiplier effects. 
This would occur in die case of import substitution; when an increase in output takes place at 



'See Califomia Economic Growdi. Center for Continuing Study of die California Economy, 
Palo Alto, Ca., 1991, pp. A1-A5, for die list of basic industries. 
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the expense of imported goods instead of locally-produced goods. If new local production 
replaces consumer or production goods and services that would have otherwise been imported 
from outside the region, then this new locally-oriented production generates additional local jobs. 
In short this import-substitution type of service activity can have multiplier effects. If all of the 
service sector training done by ETP were in businesses which competed with non-Califoraia 
suppliers, then the productivity enhancement in those industries would also be subject to a 
multiplier effect on local jobs.^ 

If we assume that all of the service sector training done by ETP is in impon substitution 
industries, the average multiplier of 1.9 would also be applied to the 35 percent of trainees 
estimated to work for service sector businesses. This is in addition to applying the multiplier to 
the 65 percent in the basic sector. Thus, the result of the import substitution assumption is that 
the multiplier is applied to all of the productivity increases generated by ETP retraining. This 
is clearly an optimistic assumption in terms of ETP*s impact, but it does suggest a criterion that 
may be useful in considering future ETP proposals. That criterion would favor proposals from 
businesses in the service sector that have potential for import substitution over those which do 
not 

Component 4: Value of Jobs Saved 

In theory, since ETP "retrainees" receive training because their jobs covered by the project 
are in jeopardy, these people should be in imminent danger of losing their jobs. The presumption 



^ The industry codes listed for the ETP projects studied here were not sufficiently refined 
to clearly identify whether the industry is export oriented or locally oriented. 
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here is that without upgrading the jobs and the woricers' skills these jobs in these particular firms 
would be eliminated The ETP trainmg upgrades the jobs and the woricers productivity 
sufficiendy to enable the firm to maintain or enhance its competitive position. 

Had ETP training not existed, the retraining and job upgrading, presumably, would not 
have occurred. These jobs could have been lost to the California economy temporarily or 
permanentiy. A temporary loss of these jobs to CaUfomia could occur while the current 
companies restructure or, if those companies retrench, until other California businesses recognize 
the market opportunity and reestablish the lost jobs. The important variable in this component 
is die time period for which the jobs are lost 

If diesc jobs are saved there arc two components to the value of die savings. First, the 
potentiaUy unemployed workers will not draw Unemployment Insurance for some period of 
unemployment because diey did not become unemployed. Second, die California economy will 
not lose die economic activity associated widi diose jobs for die period of time that they would 
have been lost in diat case. These arc savings diat would not occur in die absence of ETP. 
assuming die retrainee's jobs were lost These savings diereforc are a benefit of the ETP 
program dial should be added to die economic impact of ETP. 

There are no data on die period of time it takes an eUminated job to be reestablished. The 
proxy variable used for diat period of time is the average period of unemployment for California 
workers. These two variables ought to be very closely related if not mirror images of one- 
anodier because people have to fill reestablished jobs in oider for diem to have an economic 
impact 

To estimate die value of saving retrainee jobs from temporary loss to the California 
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economy, wc multiplied the three estimates of how long workers are typically unemployed times 
the average adjusted weekly earnings before training for retrainecs who completed. These 
estimates are presented in Scenario 4. 

There is also a very real possibility of permanent job loss. The last decade has seen a 
number of California jobs disappear, a phenomenon especially visible in the last five years. 

To estimate the value of saving retrainec jobs had permanent loss occurred, we multiplied 
the average adjusted annual earnings before training times the number of rctrainees who 
completed. The results are presented in Scenario 5. 

Estimating ETP's State-vrfde, First Year Economic Impact 

As we noted earlier, die overall first year economic impact of ETP can be thought of as 
the sum of the four components described above. The estimates for the four components vaiy 
depending on die assumptions tiiat go into each component To represent the unique contribution 
of each component, we created five scenarios for estimating ETP's overall impact. The 
assumptions that went into each scenario are described below, and actual 1990 dollar estimates 
for each scenario are presented in Table 9. 

Scenario 1: 

This scenario assumes the ETP*s impact is limited to the immediate 
savings in UI payments, and the increased productivity of trainees. 
UI payments saved are estimated by multiplying the average 
difference between UI payments to dropouts and completers tunes 
die number of completers. Productivity is simply measured by the 
actual increase in completers adjusted earnings. 
Total Impact: $ 71^54300 
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Scenario 2: 

This scenario retains the assumptions about UI savings and 
mcreased productivity. It then adds the 1.9 multiplier effect to the 
65% of increased productivity for retrainees that takes place in 
basic industries. 

Total Impact: $104^75,000 
Scenario 3: 

This scenario retains the assumption about UI savings and 
increased productivity. It then adds the 1,9 multiplier effect to all 
increased productivity for retrainees. 
Total Impact: $122,055,100 

Scenario 4: 

This scenario assumes three different periods of unemployment,as 
described before and recalculates UI savings. Adds the 1.9 
multiplier effect to all the increased productivity for retrainees. It 
also assimies that all retrainees jobs would be saved from 
temporary loss, based on the three different estimates of 
unemployment 

Total Impact: A: $332,125,700 
B: $356,971,700 
C: $437,986,900 



Scenario 5: 

This scenario assumes retrainees will be unemployed for 14.2 
weeks. Adds the 1.9 multiplier effect to all the increased 
productivity for retrainees. It also assumes that all retrainees jobs 
would be saved from permanent loss. 
Total Impact: $1,122,692,200 
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Long-term Impact of ETP Training 

ETP training continues to have an impact on the economy beyond the year immediately 
after training. While we did not study the longer term impact of ETP training, recent research 
conducted by the Rand Corporation (Lillard and Tan, 1986) indicates that training continues to 
have an impact on productivity for about twelve years. Presumably the effect of training on 
productivity diminishes over the 12 year period and disappears at the end of it 

Assuming that this pattern would hold for ETP trainees we estimated the long-term impact 
of ETP training on earnings and productivity by applying a straight-line depreciation method to 
the measured increases in productivity reported^, to estimate the effects for the following twelve 
years. This analysis assumes the impact of training will diminish 1/12 each year, and disappear 
after the twelfth year. We also adjusted these figures by discounting the estimated productivity 
to a 1990 base year through the use of a discount factor. The common practice for establishing 
a discount rate is to use a U.S. Treasury bond rate that corresponds to the period over which the 
benefits occur. Recent 12 year U.S. Treasury bond rates have been around 7.5%. So, we applied 
a discount factor of 8% to the stream of productivity increased generated by the ETP programs 
studied. 

Table 10 shows that applying this technique yields an additional impact on productivity 
of $257.2 million for years 2 to 12, when added to the year one impact the total impact on 
productivity is $323.5 million. 



^The estimate increase in productivity was the increase in earnings for the retrainees plus 76.2 percent of 
the increase in earnings for the new hires. The remaining 23.8 percent of the first year increase in earnings for the 
new hires is due to increased weeks of employment rather than increased productive; (wages). 

Does ETP Training Work? 63 



Year 

1 
2 
3 
4 
5 
6 
7 
8 
9 
10 
11 
12 

Total Years 2-12 
Total Years 1-12 
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Table 10 
Long Term Impact of 
Productivity Changes 
From ETP Training 
(8% Discount Rate) 

Impact (In Millions) 

$66.3 
54.3 
45.7 
38.0 
31.3 
25.4 
20.2 
15.5 
11.5 

8.0 

5.0 

2.3 

$257.2 
$323.5 
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Conclusions 

1. ETP training has a significant impact on the earnings of both new hire and 
retrainee completers. 

The results of this study clearly show that both new hires and retrainees who completed 
their program had significant increases in earnings that go beyond the rate inflation. Looking 
at simple differences adjusted for inflation new hire completers earned over $10,900 more than 
dropouts, and retrainee completers earned over $3,500 more than dropouts. 

When a multiple regression analysis was used to measure the impact of completing 
training after controlling for demographic and programmatic differences the impact of completing 
ETP training was still highly significant The estimated impact of completing training for new 
hires was over $7,700 and for retrainees over $4,000. 

We also calculated the simple average increase in adjusted earnings for people who 
completed ETP training. For new hires the increase was $7,400 and for retrainees it was $1,600. 
Anyway we cut it, completing ETP training had a significant and positive effect on real earnings. 

Economic theory leads us to conclude that these increased earning represent increased 
productivity for die individuals trained, and in the case of retrainees, increased productivity for 
the company that employs them. 
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2« ETP training reduces likelihood of unemployment for both new hires and 
retrainee completers. 

Trainees who completed experienced significantly less unemployment after training than 

participants who dropped out of training* Retrainccs who completed experienced about half the 

unemployment of dropouts, and new hires had on only a third as much unemployment as 

dropouts* These dramatic differences may not be caused solely by ETP training, they may be 

due to other differences between dropouts and completers such as motivation and other 

characteristics* But the fact remains that the thousands of ETP trainees who completed training 

had significantly less unemployment than those who dropped out 

3. ETP appears to have a larger impact on earning than traditional post- 
secondary vocational training offered in community colleges and proprietary 
institutions, GAIN services, JTPA Title II-A, or other state level displaced 
workers programs. 

As part of our analysis we reviewed a wide range of studies that analyzed the impact of 
public training programs on participants' earnings* These studies employed a range of research 
approaches in variety of time periods, so their results are not exacdy comparable to our analysis 
on a dollar for dollar basis* But, as we measured it, the impact of ETP training is substantially 
larger than comparable studies show for other training programs. While the exact size of the 
difference is unclear, we believe the direction of the difference is significant and reliable. Studies 
of CETA and recenUy JTPA show a consistent improvement in annual earning of only several 
hundred dollars, while both this study and a similar previous study shows the impact of ETP 
training to be several thousand dollars. 
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Part of this gap is no doubt due to the fact that ETP trains established workers who have 
the background and skills to benefit from training, while many public programs serve more 
disadvantaged participants. However we do not believe that this difference alone can account 
for the significant increases in earnings reported here. Our analysis convinces us that ETP*s 
model of employer based training with stringent performance standards is powerful training 
model that consistently produces significant impacts on both earnings and the productivity of 
workers who complete the program. 

4. ETP has a substantial impact on California's economy, beyond the increased 
earnings of participants. 

Our analysis shows that ETP training had an impact on the states economy that goes far 
beyond the increased earnings of participants. Depending on the assumptions used this impact 
ranged from $71 million to possibly over $1.1 billion in tie first year after training. ETP has 
an impact on the economy Uirough four mechanisms. First it reduces UI payments by about $5 
million dollars in the first year. Second, increased productivity, as measured by changes in 
earnings, accounts for an additional impact of over $66 million. Third, the increased productivity 
produces other economic activity within the states economy which we estimate to be between $33 
million and $51 million. Finally, assunndng that the retrainees' jobs would have been temporarily 
lost, we estimate the value of saving the jobs is between $172 million and $257 million. If 
retrainee jobs were to be permanentiy lost, the value of saving them is over $940 million a year. 

From this analysis it appears die ETP tioining has a significant impact not only on the 
productivity of individuals but also on the productivity of companies. It seems logical to infer 
that if individual productivity increases so dramatically, it likely that tlie overall productivity of 
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the company increases, which increases the competitiveness of the firm, and generates a 
multiplier effect on the economy, in many cases. 

This analysis points out that ETP is not just a training program, but, rather a powerful tool 
for economic development The issue for future ETP policy is can ETP training become a more 
effective tool by targeting its services on particular industries and companies. This analysis 
provides a ftrst step towards understanding how ETP training affects companies differently. 

5. Additional research is needed to understand the dynamics of ETP training's 
impact on companies and individuals* 

This research and the previous impact study provide a the broad outline of the impact of 
ETP training. As the economy changes and ETP policy shifts, the impact of ETP training will 
also change. It critical that ETP monitor its impacts empirically, so that the consequences of the 
new policies and the effects of shifting economic conditions can be accurately monitored. 

In addition, this study suggests the impact of training is not uniform across industries or 
company type. It's important the panel understand how the impact of training varies so that it 
can eventually target training where it will have the biggest impact 

Recommendations 

1. ETP should maintain stnngent performance standards for its contracts to 
insure that training is tied to employment 

The powerful impact of ETP on earnings and thus the overall economy are probably due 
to the powerful incentives created by the performance contracting standards. We believe 
maintaining these standards is a key to insuring the continued impact of the program. 
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2. Develop policies to increase completion rates. 

While ETP has high completion rates (78% for rctrainec and 69% for new hires) in 
comparison with similar public programs, ETP may be able to increase its completion rates. 
Since the bulk of benefits to both the individual and the economy as whole come from 
completing the program, increasing the proportion of completers will enhance the impact of the 
program. 

3, Develop targeting strategies that will lead to the greatest impact on 
California's Economy. 

Previous research and reports such as Wilms (1991) and Creticos and Sheets (1990) have 
suggested methods for targeting ETP training that arc based on the characteristics of firms and 
the need for training, or likelihood that jobs will be saved. The results of this 2tudy suggest that 
the economic impact of ETP training will vary significantly depending on the company's industry 
and the ability of training to generate productivity increases. Our analysis suggests that ETP 
could develop targeting strategies that would maximize the economic impact of ETP training. 

Here are the issues we believe the panel needs to consider in developing a targeting 
strategy. 

A major targeting issue for ETP is whether to target new hire training versus retraining 
programs. According to the research done here and the previous research on ETP, the new hire 
trainees who complete experience a change in earnings substantially higher than that of the 
retrainecs. The average change in earnings for new hires was $7,400 compared to a $1600 
average change for rctrainees in this study. These numbers would seem to make a case for 
targeting new hire programs. Before adopting a new hire program target, however, a full 
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analysis of the relative costs and relative completion rates of the two programs would have to 
be undertaken. It may be that the substantially higher benefits only come at substantially higher 
costs. Furthermore* focusing on these numbers overlooks the nature of retraining programs 
relative to new hire programs. As we noted before* retraining programs upgrade the jobs as well 
as the workers' skills* unlike the new hire program. This means that the retraining program has 
a potentially much larger impact on the California economy for every dollar increase in earnings. 
The upgraded jobs mean more productive and competitive businesses and therefore may generate 
multiplier effects within the local economy. Before targeting either the retraining or the new hire 
program* more research should be conducted to substantiate the observed change in earnings 
difference* and particularly to see if and how these differences persist over time. 

The Panel also may be tempted to de-emphasize retraining programs in durable 
manufacturing* wholesale and retail trade* and services because of the significant negative 
regression coefficients associated with these industries. Certainly a thorough comparison of the 
costs of the programs in tiiese areas should precede such a conclusion. Its important to note, 
however that* durable manufacturing is a very important basic industry, and preserved or 
expanded jobs in durable manufacturing are very likely to have multiplier effects on the 
economy, the extent of which should be known in advance of any targeting. Conversely training 
in service and retail industries has minimal multiplier effects if it does not displace imported 
goods and services. 

Our analysis and the other research point to several factors that should be considered in 
any targeting schemes: 
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Tightly-focused and well-planned retraining programs that have clear productivity 
enhancing effects should be given priority. The retraining program proposal 

should have a clear focus with goals that are cast in terms or measurable changes 
in production rates, scrap rates, rework rates, customer returns, and the like. 

Programs diat clearly enhance the competitiveness of California businesses relative 
to domestic and international competitors should be given priority. Training that 
serves businesses in the basic activity sector are likely to generate positive 
spillover effects. In these cases the relationship of the training program to the 
firm's competitive position should also be clear and measurable. 

New hire programs that train people for occupations in which the demand is 
growing at a relatively fast rate should be given priority. New hire training in 
these areas could alleviate some possible future labor market bottlenecks and such 
training has little chance of pushing people into occupations where there exists 
an adequate or excessive number of aheady skilled workers. Training in these 
latter areas would simply exacerbate unemployment rates that may be already 
rising in these occupations. 

Retraining programs diat are likely to substantially reduce the probability of layoff 
and temporarily or pcmanently lost jobs in basic industries should be given 
priority. 

4. Improve scope and quality of data collection of participant and program 
characteristics. 

A serious limit to this analysis was the fact that about 40% of the individual trainee 
records did not have the minimal demographic data currendy collected by ETP. Policies that 
insure the collection of basic participant information arc necessary for effective research in the 
future. 

In addition ETP needs collect and store electronically additional data. We particularly 
recommend that the following detail be kept: 

occupation of individuals trained as measured by a standard system such 

as the Dictionary of Occupational Titles , 

industry type by detailed SIC code (to at least three digits), 

company market position in terms of being intemadonally, nationally or 

regionally, or locally competitive. 
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specific products or services produced by employers^ 

goals of the project for productivity in^rovement, quality improvements 
or other desired change. 

5. Track the impact of ETP Training Annually. 

ETP is under a mandate to evaluate the outcomes of ETP training. This makes sense 
because, as noted before, the impact of ETP training is likely to change as the economy changes 
and ETP policies are modified This study shows that it is relatively easy to track the impact of 
ETP training by tracking the earnings and employment patterns of participants. We recommend 
that this study be replicated annually to provide policy makers and the public with an objective 
assessment of the programs impact 

6. Study the impact of ETP training over time. 

Research suggests the impact of training lasts for 12 years. Whether this holds true for 
ETP training is unknown. It is important for ETP to analyze its impact on participants and 
companies overtime. It maybe that effects are stronger and last longer in certain industries for 
certain types of participants. This type of analysis would provide the panel with sound data for 
targeting training to create long term economic benefits. 

One approach to measuring ETP training overtime would be to use the earnings and 
employment records of the 4,000 individuals who were first tracked in 1988. The initial earnings 
and background data of these participants are still archived. 

In addition the cohort of participants created for this study could be tracked annually to 
measure the continuing impact of training on individuals and the participant and program 
variables that infiuence that impact. 
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1. Ccmduct case studies of projects with exceptionally high and low changes in 
individual productivity. 

Case studies of projects where participants experienced large gains in earnings and thus 

productivity, and projects with minimal increased in earnings should be conducted to uncover the 

factors that underlie the impact of training. Again a detailed study of few projects would identify 

the characteristics of projects that produce the most powerful impact on earnings and 

productivity. It may also identify the characteristics of projects that tend to diminish the impact 

of training. These cases would provide insights into how ETP funds could be targeted most 

effectively, to provide the maximum economic impact for funds spent. 
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Appendix A: 
Comparison of Included and Excluded Cases 
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Table A-1 



Unadjusted Before and After Training 
Earnings for Included and Excluded Cases 



Before After Change 

training training 



Included $23,984 $27,413 $3,467 

Excluded 24.109 27.205 3,097 
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Table A-l 

Included and Excluded cases by Industry 



Included Excluded 





1 

. 1 


^ 1 
<. 1 


Construction 


2.4 


2.5 


Durable 
Manufacturing 


25.3 


34.3 


Non-Durable 
Manufacturing 


3.8 


4.9 


Transportation 


.3 


1.4 


Retail & 
Wholesale 


27.4 


9.6 


Finance 


16.0 


23.8 


Services 


9.8 


9.6 


Unidentifled 


14.9 


13.9 
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Appendix B: 
Regression Analysis of Unadjusted Earnings 
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Table B-1 



Regression Models For Predicting 
Change In 
Unadjusted Annual Earnings 



New Hire Retrain 
(N=l,035) (N=26,701) 
Adjusted R2=.303 Adjuscd R2=.045 



Intercept -$12,696 $4 717* 

(6,768) (ijsv) 

Complete $8,020*** 4,102*** 

Training (805) (179) 

Age -136*** -131*** 

(38) (7.61) 

Gender 

(Male) 409 1 402*** 

(930) (l63) 

Marital Status -175 .^-79 

(Married) (706) (I5O) 

Education 

< H.S. 4,125 653 

(2,760) (588) 

H.S. Grad 7,583* 1,282* 

(2,576) (508) 

Some Coll 7,250* 1,985*** 

(2,620) (514) 

Coll. Grad. 7,734* 3 i46*** 

(2,747) (522) 
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Ethnicity 



Black 1,230 -417 

(1.008) (310) 

Hispanic 1,700 -188 

(948) (197) 

Indian 3032 -572 

(2.858) (919) 

Asian 1,644 36 

(1.104) (233) 

Other 1,626 196 

(2.267) (517) 

Veteran -590 -958** 

(1.035) (257) 

Program Characteristics 
Cost Per 

Trainee 1.46 .24 

(.68) (.16) 

Number in 

Program .27 -.24* 

(.38) (.08) 

Industry 

Agriculture 5,262 NA 

Construction 12,556 737 

(6,530) (556) 

Durable Manuf. 13,087 -1,230** 

(5,593) (343) 
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Non-Durable 
Manuf. 



Transportation 
Utilities 

Wholesale/ 
Retail 



Finance, 
Insurance 
Real Estate 



Service 



Type of Skill 

Basic 



Transferable 



Unique 



12,504 
(6,332) 

16,434* 
(6,430) 



.7,952 
(5,248) 



7,853 
(15,279) 

6,175 
(5,709) 



NA 

-8,394*** 
(1,122) 

-7,754 
(5,764) 



-832 
(483) 

366 
(1.581) 



-2,118*** 
(293) 



-62 
(317) 

1,437*** 
(343) 



-3,322 

-763 
(1,559) 

-516 
(1,569) 



Occupational 



Training Objective 

Personal 
Prod. 



System Prod. 



Quality 
System 



NA 



-2,167 
(2,945) 

1,630 
(5,712) 



NA 



-106 
(1,572) 



-1,049*** 
(308) 

-1,109*** 
(269) 



NA 
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*=Significant at .01 
**=Significant at .001 
***=Significant at .0001 
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APPENDIX C: 
MULTIPLIERS 



The multipliers were developed from the IMPLAN system which consists of a set of computer 
programs, a national model of industrial production, and a massive data base containing estimates 
of final demands, final payments, production, and employment for each industrial sector for every 
county in the United States. The IMPLAN system is based on 1985 data and can provide 
estimates of interindustry purchase coefficients and the implied multipliers for any set of 
industries (up to 528 industries) and any collection of counties in the U.S. The IMPLAN system 
has been developed over the last ten years by the USDA Forest Service as an aid in land and 
resource planing. 

The multipliers reported here were developed from the IMPLAN system with the industries 
aggregated to the eight industries identified in the ETP programmatic records, and for the entire 
state of California. These multiplier are reported in TABLE C-1 for output, income, and 
employment multipliers. The output multipliers are ratios involving the dollar value of output; 
the income multipliers translate the output activity into income for California residents; and the 
employment multipliers translate the output into jobs through output-to-employment ratios. 

We used the output multipliers in our projections of the impact of ETP on the California 
economy because these ratios relate changes output in one industry to aggregate output. That is, 
these multipliers show how much California output will change as a result of a change in output 
in a given industry. Our use of the output multipliers is also consistent with our decision to use 
the most conservative basis for our estimates of ETP impact (using 1.9 instead of 2.0 or 2.1). 

We used a simple average of the output multipliers instead of applying the various industry 
multipliers to the various ETP projects because the multipliers are sufficiently close to one- 
another in value that the more complicated procedure would not add any additional accuracy to 
the economic impact projection. 
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TABLE C-1 



TYPE OF MULTIPLIER 

Sector Output Income Employment 

Agriculture L8980 1.8787 1.7392 

Forestry 

Fishing 

Mining 

Construction 1.8804 2.0465 2.1362 

Durable Mfg 1.8951 2.1632 2.2153 

Nondurable Mfg 1.8896 2.6499 2.7280 

Transportation 1.8329 1.8909 2.2754 

Utilities 

Communication 

Wholesale and 1.9827 1.9484 1.7914 

Retail Trade 

Financial 1.6816 1.6202 1.9953 

Insurance 

Real Estate 

Services 2.1026 2.0267 1.7399 

Simple Average 1.9 2.0 2.1 
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